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ABSTRACT 

The web is playing a central role in distance education. The word "usability" is usually 
synonymous with functionality of the system for the user. Also, usability of a website is 
defined as something that can be used by a specific group of people to carry out 
specific objectives in an effective way, with efficiency and satisfaction. There are some 
useful usability evaluation instruments of websites in other languages. The aim of this 
study was to develop a Persian instrument to evaluate usability in distance education 
websites. This study was a methodology research design. 

According to review literature, the main structure of evaluation instrument for usability 
designed. It was consisted of forty-eight items divided in 8 domains. Each domain was 
composed of closed multiple choice questions in which an ordinal scale with three 
values is used (always, sometimes and never) that measures the opinion of the 
evaluator about the frequency of visiting each item selected in the website. 

After that, we used face validity method. It refers to what instrument appears 
superficially to measure. At the final step, we omputed a formal content validity index 
(CVI) across the experts' ratings of each item's relevance. The content validity index 
was measured by set of ten experts, who evaluated each item individually. According 
to CVI, the final version of instrument was composed of 40 questions divided into 8 
domains: Navigation, Functionality, Feedback, Control, Language, Consistency, Error 
prevention and correction, and Visual clarity.CVI score for each phrase was more than 
0.75. According to our findings, this instrument has enough validity to apply in 
evaluation usability of educational websites of Persian distance education websites. 
However, instrument reliability can be measured in further study. 
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INTRODUCTION 


The globalization of distance education provides many opportunities for developing 
countries for the realization of their education system-wide goals. The terms open 
learning and distance education represent approaches that focus on opening access to 
education and training provision, excommunication of learners from the constraints of 
time and place, and offering flexible learning opportunities to individuals and groups of 
learners. 

Open and distance learning is one of the most rapidly growing fields of education, and 
its potential impact on all education delivery systems has been greatly accentuated 
through the development of I nternet-based information technologies, and in particular 
the World Wide Web (UNESCO, 2002). Today many institutions in different societies 
offer distance education or online degrees to individuals who seek opportunities for 
life-long learning. Distance education (DE) constitutes an important and integral 
teaching/ learning methodology, one that many feel will support emerging growth 
trends in education and training worldwide. Various forms of DE delivery, particularly 
those based on the use of information and communication technology (ICT) tend to be 
included in the operational scope of many higher education institutes. Universities 
oriented towards the use of open and distance learning were first in-line when it came 
to adopting new and emerging technologies, actions that heralded an new era of 
increasing access to further education and training (Goho, MacAskill, & McGeachie, 
2003). 

Distance education has involved a more sophisticated technology, moving towards 
virtual environments in which instruction from a host site is distributed to distant sites 
by using a combination of live, two-way interactive audio, video, or both, and 
synchronous and asynchronous computer-based interactions that use local area 
networks (LANs), wide area networks (WANs), and the Internet (Williams, Pabrock, & 
Covington, 1999). The web is playing a central role in distance education. While 
distance education continues to expand by using online or I nternet-based 
technologies, it is important to recognize learners as clients of education and to be 
concerned about their attitudes toward internet-based learning as well as their 
satisfaction with educational services provided (Mayzer & Dejong, 2003). 

The word "usability" is usually synonymous with functionality of the system for the 
user. Also, usability of a website is defined as something that can be used by a specific 
group of people to carry out specific objectives in an effective way, with efficiency and 
satisfaction. In other words, usability is the property of a mediated learning 
environment of supporting the users as transparently as possible in the 
accomplishment of their learning goals (Triacca, et al., 2004). 

If we assume that the organization and the usability of the information is partly 
responsible for a better learning (Najjar, 1996), then a usable e-learning website is not 
just a resource with a nice "look & feel", but a web application which communicates 
contents and structures the interaction in such a way that facilitates the learning 
experience. However, it is clear that usability evaluation is just one specific aspect of 
the quality assessment of a e-learning environment. A highly usable online course does 
not guarantee at all a high quality in the learning outcome (Triacca, et al., 2004). 
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How can usability of an e-learning web application be effectively evaluated and 
measured? How can evaluators, instructors, reviewers, tutors and course managers be 
equipped with usable and ready-to-use tools for evaluating the usability of the web 
application prototyped or deployed? There are some useful usability evaluation 
instruments of websites in other languages. 

This paper presents an adapted version of a proven methodology for usability 
evaluation which has been already used in a variety of web application domains and 
that has been used and tailored for e-learning web applications. The aim of this study 
was to develop a Persian instrument to evaluate usability in distance education 
websites. 

METHODS 

This study has done by methodologic research design. Methodologic research refers to 
investigations of the ways of obtaining, organizing, and analyzing data. Methodologic 
studies address the development, validation, and evaluation of research tools or 
methods. 

Most methodologic studies are descriptive and nonexperimental, often focusing on 
instrument development and testing (Polit & Beck, 2003). Stages of this research work 
were: 

• First stage to determine the definition of "evaluation of educational Web site": 
Referring to texts and sources in Persian, the theoretical and practical definition of 
this relationship not found. The common issues found in the literature relating to Web 
site evaluation are quality, Web design, and usability. Researchers have adopted web 
quality concept from the quality of product or service. For example, some adopted 
Kano's Model of Quality as a theoretical framework to evaluate the quality of Web sites 
(Hung & McQueen, 2004). 

• The second stage to determine the domains and phrases: 

According to review literature, the main domains of usability evaluation instrument 
were extracted (Hung & McQueen, 2004; Triacca, et al., 2004; Matera & al, 2002; 
Masterilli, 2009; Erica & Sherae, 2007). 

These main domains consisted of Navigation, Functionality, Control, Language, 
Feedback, Consistency, Error prevention and correction, Visual clarity. Each domain 
was composed of closed multiple choice questions in which an ordinal scale with three 
values is used (always, sometimes and never) that measures the opinion of the 
evaluator about the frequency of visiting each item selected in the website. 

• The third stage to determine tool validity: 

We used face validity and content validity methods to validity determination of 
instrument. Face validity refers to whether the instrument looks as though it is 
measuring the appropriate construct. Content validity concerns the degree to which an 
instrument has an appropriate sample of items for the construct being measured. It 
was done by using a panel of experts to review the items. 

At the final step, an instrument's content validity is necessarily based on judgment; we 
computed a formal content validity index (CVI) across the experts' ratings of each 
item's relevance (Yaghmaie, 2003). 
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The content validity index was measured by set of ten experts, who evaluated each 
item individually. A Part of the questionnaire that show to determine how content 
validity index will follow in table 1: 


Table: 1 

A part of the questionnaire to assess content validity by CVI 
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There is a clear indication of 
the current location 


X 



X 






X 


2 

All major parts of the site are 
accessible from the Home 
page 
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Site structure is simple, with 
no unnecessary levels 
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Relevance: 

1. Phrase is not related. (Is irrelevant) 

2. Phrase is to some extent the need to review. 

3. Phrase is related, but has little need to review. 

4. Phrase is very relevant. 

Be clear: 

1. Phrase is not clear. (Is unclear) 

2. Phrase is to some extent the need to review. 

3. Phrase is clear, but has little need to review. 

4. Phrase is very clear. 

Be simple: 

1. Phrase is not easy. (Complex) 

2. Phrase is to some extent the need to review. 

3. Phrase is simple, but has little need to review. 

4. Phrase is very simple. 

RESULTS 

Content validity method is a qualitative manner. But in quantitative manner, CVI 
procedure is to have experts' rate items on a four-point scale. Each phrase points to be 
derived from three fields of relevance, clarity and simplicity. This range was at least 3 
points up to 12. I n other words, if there is maximum value of fields, the final point for 
that phrase will be 12 points. According to number of expert panelist, the average point 
of each phrase is extracted. I f the average points obtain more than 75 percent of 12, it 
is acceptable content validity index. Since the 8 phrases were obtained less point than 
0.75, therefore, only CVI of 40 phrases were confirmed. According to CVI, the final 
version of instrument was composed of 40 phrases divided into 8 domains: Navigation 
(included 6 phrases), Functionality (included 4 phrases), Control (included 6 phrases), 
Language (included 4 phrases), Feedback (included 6 phrases), Consistency (included 4 
phrases), Error prevention and correction (included 5 phrases), and Visual clarity 
(included 5 phrases). CVI score for each phrase was more than 0.75. 
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Since the responses consisted of three words per condition "never", "sometimes", and 
"always" is therefore allocated the contract rates from zero to 2, and in the best 
conditions with the total evaluation points will be 80 of the numeration decimal system 
(percentage basis) the maximum equivalent of 100 is considered. From other 
contractual allocation as the third point of cutting rates can be assessed to the 33 poor, 
33 to 66 average evaluation, and greater than 66 categories to be desired. 

CONCLUSION 


Many articles on website evaluation criteria are available. However, access to native 
tools on appropriate to each country's culture and technology are essential. I n order to 
develop this tool, initially based on library studies, evaluation tool was written, and 
then use of experts opinion, content validity was reviewed and approved. Therefore, 
this tool can be useful as an evaluation tool with practical approach for evaluating the 
usability of distance education websites. However, perhaps due to common strategies 
in websites designing, some phrases of this tool may be applicable in evaluation of 
non-educational types of websites, however, researcher has focused and purposed in 
determining educational websites. 

Regarding the development of distance education system in I ran, and the necessity of 
education website evaluation, these tools can be used as a first step in this way. Future 
work will consolidate the results of the usability evaluation by application of this tool. 
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